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ABSTRACT 



Using British research related to the Open University's 
Postgraduate Certificate in Education (OU PGCE) , a preservice teacher 
education program, this paper considers how open and distance learning (ODL) 
programs are evolving quality assurance systems and procedures across a range 
of elements, including student teacher assessment. The paper explores how 
high-quality provision and outcome standards of students are assured in a 
high-volume, highly dispersed program with over 2,000 school-based contexts 
for practice each year. Section 1 presents an overview of some ODL teacher 
education programs. Section 2 discusses quality assurance issues for ODL 
preservice teacher education. Section 3 focuses on assessment (an example of 
high quality in an open learning preservice education program) . Section 4 
examines issues related to improving program quality. The paper concludes 
that three factors (course materials with their common structure; the 
competence model with its open, shared, and moderated assessment; and the 
program support and monitoring network of the Open University) all combine to 
maintain the quality and integrity of the course and qualification. An 
appendix presents an OU PGCE competence and professional qualities model . 
(Contains 7 figures and 20 references.) (SM) 
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Introduction 

Who would argue against the proposition that all school pupils have a right to be taught by 
competent teachers irrespective of where and how those teachers were educated and 
trained? By 'competent 7 , we mean that teachers have the necessary and sufficient subject 
and pedagogical knowledge to promote pupil leaning at the highest level and are able to 
demonstrate this through effective teaching in the classroom. Yet there seems to be many 
obstacles to the achievement of this worthy aim : 

• not all who subscribe to the above aim would agree on how the descriptors of 
'competent' and 'sufficient' should be defined as a valid description of the complex 
teachers' role; 

• even accepting the criteria of what may constitute competence, can those criteria be 
assessed reliably in the different school environments and multiple contexts in which 
pre-service student teachers work? 

• and given that an institution may indeed settle on its own valid and reliable assessment 
procedures, would those procedures be acceptable to other teacher educators working in 
different institutions, in a different economic and political context and with their own 
particular notion of what is a desirable and acceptable teacher performance for entry to 
the teaching profession? 

Jaap Tuinman, the ex-Dean of the Faculty of Education at Simon Fraser University is critical 
of the prospects of the higher education community coming together, even within just one 
department, to agree on answers to some of the above questions: 

"The unwillingness of education faculty members to agree on content and approach 
to a particular course makes a shambles of the concept of a university curriculum 
and a mockery of the idea that teacher education entails the preparation for a 
profession" (Tuinman, 1995 pll4) 

In the United Kingdom, the government has taken a lead in specifying, with ever greater 
delineation, the criteria for what constitutes competence, specifically identifying standards 
that all newly qualified teachers must demonstrate (see Macintyre, 1991, DFE, 1992, DFEE, 
1997). Recently (TTA 1998) there has been a move to specify input as well as outcomes in 
defining a national curriculum for pre-service teacher education. 

This paper, drawing on research and development relating to the Open University’s 
Postgraduate Certificate in Education (OU PGCE), a pre-service teacher education 
programme, considers how open and distance learning programmes are evolving quality 
assurance systems and procedures across a range of programme elements, including 
assessment of student teachers. In particular, it explores how high quality provision and 
outcome standards of students are assured on a high volume, highly dispersed programme 
with annually over 2000 school-based contexts for practice. 

There may be lessons for how we may move towards valid and reliable assessment of 
teacher performance in a range of different teacher education contexts, in various types of 
schools and in the variety of education systems around the world. 

A brief overview of some open and distance learning teacher education 
programmes. 

Teacher education open and distance learning (ODL) programmes have a long and 
successful history but it is only in recent years that the techniques have been applied to the 
preparation of pre-service teachers and the complexities of assessment of school-based 
practice. 
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University-level distance learning institutions were established in the 1930s, using 
correspondence tuition. Many teachers used these programmes to upgrade their subject 
knowledge, but in the early 1970s with a dramatic increase in 'Open University' institutions 
using a new multiple-media approach, student numbers soared. For example, the UK Open 
University began undergraduate education programmes in 1971 and classroom teachers 
reached a level of 40% (24 000 students) of the total undergraduate cohort as they converted 
their college teaching diplomas into first degrees with a consequent increase in salary. Such 
undergraduate courses, through their emphasis on subject and educational theory, avoided 
the complexity of assessment of classroom teaching. Open and distance learning in-service 
programmes were particularly popular in developing countries such as Zimbabwe 
(Matshazi, 1992), Kenya (Odumbe, 1992), Pakistan (Robinson, 1993). The Chinese Television 
Teachers College in the early 1990s was supporting 200 000 teachers per year to up-grade 
their qualifications (McCormick, 1992). More than 40 developing countries have established 
ODL programmes to update under qualified teachers. Such in-service courses continue to 
be an extremely important means of providing cost-effective teacher education in both the 
developed and developing world (Leach & Lita 1996, Hobbs et al 1997, Moon 1997). 

However, these courses were similar to the undergraduate courses of earlier years in that 
their assessment strategy did not place the school as the principalsite for learning and 
assessment. 

In February 1994, the first cohort of 1000 student-teachers began their studies for a new 
national, open and distance learning, pre-service teacher training and education programme 
at Britain's Open University. Assessment is premised on a competence-based model and 
therefore the school becomes a key player for training and assessment. 

Thus, in this short overview, three generations of open and distance learning teacher 
education models can be determined. The first generation provided tuition in general 
educational theory assessed in isolation from the classroom. The second generation drew 
on classroom practice to inform the assessment of 'projects' related to the practice of a 
school teacher. The third generation courses, significantly, use the school as a site for 
learning and require a combination of assessment procedures to assess knowledge, 
understanding and also competence in the practice of teaching. (Moon, 1996) 

But, can mass education programmes using ODL methodology be of high quality? Do the 
teachers improve their knowledge, understanding and teaching skills? Evaluation studies 
around the world are very encouraging: 

If success in written examinations is accepted (as it is for all professions) as evidence 
of improved knowledge, then distance in-service training is very effective 
(Hawkridge, 1993) 

When first confronted with distance education it is only natural to wonder whether 
its quality is equal to that of ordinary college or university. [...] Mid-Sweden college 
has made a comparison of the completion rates of traditional university education 
and distance programmes in the municipalities in the region. [...] the rate was 
shown to be equally high for ordinary and distance students - in some cases the 
latter was higher. (Asplund and Bjorne, 1995) 

Quality Assurance: Issues for ODL pre-service teacher education 

The Open University's Post Graduate Certificate in Education (OU PGCE) is an example of 
the third generation of teacher education models. A brief outline of the OU PGCE 
programme will set the scene for the quality assurance issues that need to be addressed. 

The OU PGCE programme was designed to provide a new pre-service teacher education 
route that would provide access to qualified teacher status for new sections of society, 
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primarily mature graduates embarking on a second career. It was also designed to deliver 
high student numbers (7.2% of national graduate pre-service training in 1996) particularly in 
’shortage' subjects such as science and mathematics, cost effectively. 

The programme structure was therefore designed as part-time (18 months) home-based 
study using materials in a range of media and requiring school-based practice in local 
schools. The model operates nationally with a regional infrastructure and local tutor-based 
support. 

The programme statistics are as follows: The programme began in 1994 with the first 

students graduating as qualified teachers in 1995. Each year 1000 new students are 
recruited to nine subject/phase specialisms. School-based practice takes place generally on 
a one student: one partner school basis. Each year the programme works with 1000 different 
schools, with 1000 mentors and 1000 school-based co-assessors. 70% of schools and school 
staff are new to the programme each year. For 6 months of the year there is an overlap 
between two cohorts doubling student and school numbers. To date over 3000 teachers 
students have qualified to teach on the programme. 

The quality assurance procedures developed need to assure high quality provision and 
outcome standards across all the components of the programme (see fig. 1) and provide 
information for continual improvement ie procedures serve a monitoring and evaluative 
function. The scale of the programme and the variability of school contexts for training and 
assessment are the critical issues to address in developing such procedures. 

Fig 1 Quality assessment across all course components 
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The quality assurance procedures linked to the programme have been refined through a 
continual process of evaluative research leading to the establishment of a set of key 
principles for assuring high quality in open and distance learning programmes. These 
principles are: 

• explicit outcomes 

• prescribed common frameworks 

• triangulated evidence 

• interconnected procedures 

• systematic monitoring -indirect, direct and in response to structural 'triggers' 

These principles, we believe, can contribute to a wider debate on methods to ensure high 
quality in pre-service teacher education. 

Assessment: an example of high quality in an open learning pre-service 
education programme 

Berliner (1988) and Furlong et al (1988) have described a progression from 'novice' to 
'expert' which has influenced the way the OU PGCE programme team designed the 
structure of its pre-service programme. 

Progressively, students move from the position of observer and helper of experienced 
practitioners, through collaborative, teaching towards solo teaching. 




5 



D 



In forming the assessment policy, the programme team drew extensively from the work on 
teacher competence assessment (for example Whitty and Willmott (1991), Harvard and 
Dunne (1992), Moon and Shelton Mayes (1993)) However, many elements of the Open 
University PGCE were developed in consultation with members of the profession itself and 
this was particularly the case for the assessment model. Groups of teachers contributed to 
its structure, including some of the first school mentors required to H operationalise , ' the 
concept. The assessment strategy is based on a set of competences’ and professional 
qualities’ and students are required to provide evidence, in their portfolio, demonstrating 
each element of both components by the end of the course. 

The OU PGCE programme team drew on the principles outline above to develop a rigorous 
model of assessment. In doing so the programme team has access to extensive data that is 
used to inform development. 

• explicit outcomes. The adoption of a competence -based assessment model provides an 
explicit statement of outcomes for students and assessors; for self-assessment, formative 
and summative assessment. The OU PGCE programme team took a wide definition of 
competence, following Whitty and Willmott (1991): 

"competence is wider than merely an ability to perform a task satisfactorily, 
encompassing intellectual, cognitive and attitudinal dimensions as well as 
performance" 

We would also concur with their observations that competence approaches may have a 
number of benefits: demystification of teacher education; a clearer role for the partners in 
the training process; greater confidence of employers in what beginning teachers can do; 
and clearer goals for the students. Critically, for a high volume, widely dispersed 
programme, an explicit assessment outcomes model provides a shared set of standards 
underpinning all assessment processes. 

The Open University PGCE describes the teaching process in terms of five area of teaching 
competence: 

A Curriculum/subject planning and evaluation; 

B Classroom/subject methods; 

C Classroom management; 

D Assessment, recording and reporting; 

E The wider role of the teacher. 

These areas are further divided into about 5 subcategories. For example: 

A2 identify diversity of pupil need in the context of appropriate strategies for ensuring 
continuity and progression. 

Taken overall, the framework describes 22 elements of competence. However, the 
programme team considers that teaching is not only concerned with exhibiting certain 
teaching competences. It is also necessary that the competences exist within a framework of 
the professional qualities appropriate to the teaching profession. Evidence of teaching 
competence, should also illustrate professional qualities such as: commitment to 
professional values; effective communication; appropriate relationships; effective 
management. Both teaching competences and professional qualities are explicitly set out as 
assessment outcomes. This ensures both dimensions of assessment drive formative 
assessment and hence training, as well as summative assessment. 

• common prescribed framework. The variability of school context and national coverage 
requires a tightly prescribed framework that all involved in training and assessment 'sign 
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up' to. This provides an entitlement to training and assessment for students and sets the 
criteria by which internal and external monitoring is carried out. 

The common prescribed framework extends to: centrally produced but regionally delivered 
training programmes for assessors; a distance learning "Mentor Training Programme" which 
supports the mentor and the school co-assessor; an assessment reporting framework for 
mentors and tutors; detailed school-based assessment activities structured through common 
school experience guides and assessment guides; and a structured professional development 
portfolio of evidence structured by the competence and professional qualities model. 

The production of common and published text materials ensures a consistency of approach 
from year to year. We agree with Tuinman (1995, pll4): 

"To use claims of academic freedom [by education faculty] in order to escape what is in 
effect an implicit contract to teach a certain segment of the university curriculum is not 
acceptable". 

• triangulated evidence. Assessment, formative and summative, is carried out by 
students, tutor and school staff . Student self-assessment against the competences are 
submitted alongside school-based assessment and tutor assessment. All school-based 
evidence is assessed by mentor and a senior member of staff acting as internal co- 
assessor. A prescribed evidence base to support assessment judgements is required from 
students and assessors. All sources of evidence and judgements are cross-referenced 
within monitoring undertaken by programme team. 

• comprehensive monitoring. Assuring the standard of student assessment is achieved by 
interconnecting direct and indirect modes of monitoring. 

There is a range of complementary, indirect monitoring. All schools reports and tutor 
reports are monitored by the programme team for : compliance with the common 
prescribed assessment guidance; grading accuracy; match with evidence; and quality of 
information. The outcome of the indirect monitoring is a critical indicator in initiating direct 
monitoring and continual tracking of individual students, schools, tutors. 

For example: 

• Unsatisfactory grades on tutor-marked assignments is an important indicator of possible 
school-based weakness as the assignments are specifically designed to integrate school- 
based experience with theory. Fig 2 illustrates programme monitoring of tutor-marked 
assignments for a single cohort (1997). 

• Non- attendance at mentor briefing for assessment, results in additional direct school 
visits (Attendance was 67% nationally in 1997) 

• Grading inaccuracy by tutors and mentors focuses staff development. 

• Unsatisfactory student gradings leads to progress review sessions for students with 
supplementary mid-course support or, if necessary, de-registration. 
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fig. 2 
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Reviewing outcomes from this range of indirect and interconnected monitoring leads to 
prioritisation of direct monitoring which is undertaken by programme team working in 
regions. The use of systematic indirect monitoring to drive direct monitoring is critical in 
assuring the quality of a large, geographically dispersed, student and school population. 

Alongside this systematic indirect and direct monitoring there is random direct monitoring 
of tutors, schools and students. An analysis of random direct monitoring is used to confirm 
the adequacy of systematic procedures. 

Finally, external moderation of student outcomes by external examiners completes the range 
of quality assurance procedures. 

An analysis of student assessment outcomes (see fig 3) year on year show changing numbers 
of students in categories: fail, pending, withdrawal, which can be examined against changes 
in procedures e.g. the introduction of de~ registration procedures. This provides evidence 
that quality assurance procedures are operating appropriately during the course to identify 
students making limited progress against the assessment criteria. 



fig. 3 Student assessment outcomes 




Quality Issues : Improving the quality of the programme 

The extensive data collected year on year in relation to assessment is used, not only to 
assure quality of student outcome standards, but significantly to identify areas for 
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development of the programme. Most commonly this leads to internal refinements in 
procedures, but the extent of the data available from a large population raise wider issues 
for assessment of teacher quality. 

A brief example of refinement to programme procedures is: 

• In 1995, an analysis of 1000 summative school reports led to changes in assessment 
strategy focusing on staff development relating to target setting for career entry 
induction. 

However, an example of the potential of the data to inform a wider debate follows from an 
analysis of data on summative assessment of students against the competence model: 

• All students receive summative assessment profiles completed by tutors and mentors 
identifying concerns for the individual student in relation to the 22 competence sub- 
categories. The analysis of the summative profiles is used by the programme team to 
assure standards in final assessment i.e. any concerns identified by the mentor and tutor 
form the basis of subsequent additional assessment by faculty staff for those identified 
individuals. But a subsequent analysis of 'concerns' for the total cohort (see figs. 4, 5, 6 
and 7 ) yields rich data on the differences of assessment perceptions between the 
different players. In this case, analysis indicates that mentors' concerns lie in those 
competences linked to the development of methods for teaching strategies; that tutor and 
mentor patterns of concern differ; and there are differences in assessment performance 
between student- teachers of different subjects. The thrust of development of the 
programme is therefore directed to addressing this issue, reviewing training and 
assessment guidance for mentors and student curriculum in these specific areas. 



fig 4 Final assessment concerns flagged in each area of competence (See Appendix for key) 
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fig 5 Final School Mentor assessment concerns in each sub-element (See Appendix for key) 
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fig 6 Final OU Tutor Concerns in each sub-element (See Appendix for key) 
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fig 7 Comparing final assessment concerns for Science and Technology student-teachers 
with those for History and English student- teachers. 
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Conclusion 

The course materials with their common structure; the competence model with its open, 
shared and moderated assessment; and the programme support and monitoring network of 
the Open University all combine to maintain the quality and integrity of the course and 
qualification. In addition, the procedures for ensuring quality deliver a rich database for 
evaluative research linked to ongoing development of the programme. 

• Development work with thousands of teachers nationwide and across subjects helps to 
create a shared set of criteria of what constitutes successful teaching. By continued 
dialogue in mentor briefings a consensus emerges which increases validity in teacher 
assessment. 

• The very open nature of the competence criteria, shared by school mentor, Open 
University tutor, faculty staff and student helps to satisfy concerns over reliability of 
judgement. 

By these methods the thousands of students qualifying to teach through the programme, 
working in a range of contects throughout the UK and parts of Continental Western Europe 
can be assured of a high quality experience wherever they are located. The indirect and 
direct assessment procedures combine to assure only teachers who "have the necessary and 
sufficient subject and pedagogic knowledge" receive qualified teacher status. 
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Appendix 

OU PGCE Competence and professional qualities 

model 

Teaching Competences 

A Curriculum/subject planning and evaluation 

Demonstrates an ability to: 

A I apply subject and curriculum knowledge (including national curriculum ) appropriate 
for the subject and whole curriculum. 

A2 identify diversity of pupil needs in the context of appropriate strategies ( methods ) for 
ensuring continuity and progression 

A3 plan and critically evaluate at the level of pupil activity, a lesson , sequence of lessons 
and scheme of work, for whole class, groups and individuals, with due regard to how 
pupils develop and learn. 

B Classroom/subject methods 

Demonstrates an ability to: 

Bl implement a range of teaching and learning strategies appropriate to pupil needs and 
tasks, including whole class, group and individual set tasks. 

B2 respond flexibly to the needs of pupils and classroom circumstances. 

B3 use language and other means of presentation in a clear and stimulating manner at 
individual, group and class level. 

B4 motivate pupils and maximise potential 

B5 support pupils in developing cross-curricular dimension and skills. 

C Classroom management 

Demonstrates an ability to: 

Cl establish a physical environment suitable for teaching and learning. 

C2 create a social environment conducive to teaching and learning. 

C3 organise classroom time effectively 
C4 manage resources appropriately, including IT. 

D Assessment, recording and reporting 

Demonstrates an ability to: 

D1 select and implement appropriate strategies and systems for formative assessment. 

D2 select and implement all aspects of summative assessment relevant to the task. 

D3 promote pupils' capacity for self and peer assessment and evaluation 
D4 record and report effectively 
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E The wider role of the teacher 

Demonstrates an ability to: 



El play a full role in teaching teams. 

E2 play a full role in the life of the school. 

E3 support pupils in their personal , social , spiritual , moral and cultural development. 

E4 provide effective partnerships with parents and governors 
E5 liaise effectively within the wider school community. 

Professional qualities 

Commitment to professional values 

Teachers demonstrate this quality by personal example and through their role in school by, 
for example: 

• respecting and valuing pupils as individuals in order to promote personal growth and 
autonomy; 

• acknowledging their own role and responsibilities and the roles and rights of other 
individuals and groups in the educational process; 

• understanding and implementing equal opportunities principles and practices; 

• managing and resolving complex ethical responsibilities and value conflicts; 

• engaging creatively in continuing professional development, including self-evaluation, 
recognising one’s strengths and limitations; 

• contributing to school and wider debate about school development and improvement. 

Effective communication 

Teachers demonstrate this quality by personal example and through their role in school by, 
for example: 

• communicating in a form and manner which is clear, sensitive, varied in style and 
medium, and appropriate to different audiences and purposes. 

Appropriate relationships 

Teachers demonstrate this quality by personal example and through their role in school by, 
for example: 

• developing collaborative relationships with pupils, parents, colleagues and other 
professionals; 

• showing the ability to empower others. 

Effective management 

Teachers demonstrate this quality by personal example and through their role in school by, 
for example: 

• showing an ability to act in independent manner, use initiative, and to prioritise. 
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